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Abstract 

We consider the stable approximation of sparse solutions to non-linear 
operator equations by means of Tikhonov regularization with a subqua- 
dratic penalty term. Imposing certain assumptions, which for a linear 
operator are equivalent to the standard range condition, we derive the 
usual convergence rate O(Vo) of the regularized solutions in dependence 
of the noise level 8. Particular emphasis lies on the case, where the true 
solution is known to have a sparse representation in a given basis. In 
this case, if the differential of the operator satisfies a certain injectivity 
condition, we can show that the actual convergence rate improves up to 
0(5). 

MSC: 65J20; 65J22, 49N45. 



1 Introduction 

A widely used technique for the approximate solution of an ill-posed, possibly 
non-linear operator equation 

F(u)=v (1) 

on a Hilbert space U is Tikhonov regularization, which can be formulated as 
minimization of the functional 

T(u) = \\F(u) -v\\ 2 + aK(u) . 

The first term ensures that the minimizer u a will indeed approximately solve 
the equation, while the second term stabilizes the process of inverting F and 
forces u a to satisfy certain regularity properties incorporated into 1Z. Originally, 
Tikhonov applied this method to the stable solution of the Fredholm equation. 
Requiring differentiability of u a , he used the square of a higher order weighted 
Sobolev norm as penalty term [HI [37J . 

Recently, the focus has shifted from the postulation of differentiability prop- 
erties to sparsity constraints [101 13 13 HI IE [HI [HI HZl HE I2H1 HI] • Here, one re- 
quires the expansion of u a with respect to some given orthonormal basis (</>j)ieN 
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of U to be sparse in the sense that only finitely many coefficients are different 
from zero. This can be achieved with regularization functionals 



In fact, sparsity of the solution is not necessarily guaranteed for q > 1. The 
lack of convexity of 1Z, however, makes a choice q < 1 inconvenient both for 
theoretical analysis and the actual computation of a minimizer. On the other 
hand, the assumption q < 2 is used to obtain coercivity of the regularization 
functional, which in turn implies the existence of minimizers of T. For these 
reasons we only consider the case 1 < q < 2. 

We concentrate our analysis on the well-posedness of the regularization 
method and the derivation of convergence rates. For that purpose we assume 
that only noisy data v s is given, which satisfies \\v s — v\\ < 8. We denote by u s a 
the minimizer of the regularization functional with noisy data v s and regular- 
ization parameter a, and by an 7£ 9 -minimizing solution of F(u) — v. Then 
the question is how the distance \\u s a — u^W depends on the noise level 8 and the 
regularization parameter a. 

Dismissing for the moment the assumption of sparsity, we derive for a param- 
eter choice a ^ 8 & convergence rate vJ\\ — 0(V8) provided 1 < q < 2 and 
a source condition is satisfied (see Proposition [T2"|) . In the linear case this condi- 
tion is the usual range condition d1Z(u>) [Hrange(F*) ^ 0, where F* denotes the 
adjoint of the operator F (see Propositionllip. Similar results have been derived 
recently [20] [22]. In the non- linear case we impose a different assumption, which 
for sparsity regularization generalizes common source conditions involving the 
Bregman distance [15 1 125 1 125]. 

If, furthermore, the solution u' of the operator equation is known to be 
sparse, then the convergence rates of the regularized solutions to can be 
shown to be 0(S 1 ^ q ) where 1 < q < 2 is the exponent in the regularization 
term ([2]) (see Theorems [14] and [15]) . To that end we require the derivative of 
F at to be invertible on certain finite dimensional subspaces, a condition 
introduced in [5] for linear operators as 'finite basis injectivity property'. This 
improved convergence rate provides a theoretical justification for the usage of 
subquadratic penalty terms for regularization with sparsity constraints. 

Our results reveal a fundamental difference between quadratic and non- 
quadratic Tikhonov regularization. Neubauer [21] has derived a saturation re- 
sult for quadratic regularization in a Hilbert space setting with a linear operator 
F. He has shown that, apart from the trivial case u' = 0, the convergence rates 
cannot be better than 0(8 2 ^ 3 ). The present article shows that this rate can be 
beaten by sparse regularization when applied to the recovery of sparse data. 

2 Notational Preliminaries 

All along this paper we assume that V is a reflexive Banach space and U is 
a Hilbert space in which a frame (</>i)igN C U is given. That is, there exist 
< Ci < C 2 < oo such that 




< q < 2 . 



(2) 



CilMI 2 <£l<<MI 2 <c< 2 |M| 2 



for every u <EU . 
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The operator F: dom(F) C U — > V is assumed to be weakly sequentially closed 
and dom(F) ndom(7?. 9 ) ^ 0. Examples for weakly sequentially closed operators 
arc linear bounded operators restricted to convex domains, which naturally arise 
for instance in image restoration problems or tomographic applications |15[ I25j . 
Truly nonlinear operators arise in schlieren imaging |25j or simultaneous activity 
and attenuation reconstruction in emission tomography [T2] . See also [TJ H3] for 
the application of sparsity constraints to inverse problems. 

We define the regularization functional lZ q : U — > R U {00} by 

where 1 < q < 2 and there exists u> m i n > such that Wi > w m in for all i £ N. 
Note that lZ q is convex and weakly lower semi-continuous as the sum of non- 
negative convex and weakly continuous functionals. 

The subdifferential of lZ q at u is denoted by d1Z q (u) C U, If q > 1, then 
dlZ q (u) is at most single valued and is identified with its single clement. 

For the approximate solution of the operator equation F(u) = v we consider 
the minimization of the regularization functional 



\F(u) - v\\ P + aTZ q (u) , if u G dom(F) n &om.{K q ) . 
f 00 , if u $ dom(F) n dom(7?.q) . 



with some a > and p > 1. 

In order to prove convergence rates results we impose an additional as- 
sumption concerning the interaction of F and lZ q in a neighborhood of an 
7£ g -minimizing solution of F(u) = v. Here w G U is called 7?. g -minimizing 
solution, if F(vJ) = v and 

Tl q (u ] ) = mm{TZ q (u) : F(u) = v} . 

Assumption 1. The equation F(u) — v has an H q -minimizing solution xv and 
there exist fix, 02 > 0, r > 0, a > 0, and p > lZ q (v)) such that 

TZ q (u) - n q (ui) > Px\\u - v)f - (3 2 \\F(u) - F(u^)\\ (3) 

for all u G dom(i 7 ') satisfying lZ q (u) < p and \\F(u) — F(u*)\\ < a. 

In Section 0] below we show that Assumption [1] with r = 2 follows from 
the standard conditions stated in general convergence rates results in a Banach 
space setting [5J [21] , which in turn generalize the standard conditions in a 
Hilbert space setting [15j [16] . Moreover, the assumption is equivalent to the 
standard source condition d1Z{v)) n ranged*) ^ in the particular case of a 
linear and bounded operator F (see Proposition ITTj). 



3 Well-Posedness and Convergence Rates 

In this section we prove the well-posedness of the regularization method. By 
this we mean that minimizers u s n of the regularization functional T v ' q * exist for 

every a > 0, continuously depend on the data v s , and converge to a solution 
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of F(u) — v as the noise level approaches zero, provided the regularization 
parameter a is chosen appropriately. 

These results are analogous to results obtained for standard quadratic Ti- 
khonov regularization in Hilbert spaces (see e.g. [IS]). Also the mathematical 
techniques employed in the proofs of existence, weak stability, and convergence 
are similar. Some extra work is needed, however, for the passage from weak 
stability and convergence to stability and convergence with respect to 7Z q . 

Lemma 2. Let 1 < q < 2. Assume that (uk)keN C U weakly converges to u £ Z7 
and that TZ q (uk) converges to lZ q (u). Then lZ q (iik — u) — ► 0. 

Proof. The assumption 1Z q (uk) — > 1Z(u) implies that 



lim sup TZ q (uk — u) 

k 



lim sup 

k 



2(ll q (u k ) + K q {u)) - 2(K q {u k ) + K q {u)) + K q (u k - u 
4K q (u) - liminf ^ Wi [2 | u k ) \ q + 2 |(&, u)\ q - | - u) \" 



Using Fatou's Lemma we obtain that 

- hminf J2 w i[ 2 \ I" + 2 I u > I" " I (&> ^k - u) |« 

"I 

< - E liminf [2 I <^,it fe )| <? + 2 I u) | 9 - K^i,« fc - u) | 9 



f'tK 



Now, the weak convergence of (u k )keN shows that {4>i,Uk) ~ ■* (4>i,u) for all 
i£N. Therefore it follows that 

- E lim fc inf ^ [2 I (0, , «fe) I 9 + 2 I m) | 9 - I , u fe - w) | 9 ] = -4 ^ I (0, , u) | 9 . 

ieN ieN 

Combining the above inequality and equalities we see that 

lim sup 72,(1** -u) < 472 9 (w) -4^ Wl |(^, u)| 9 = 



or, equivalently, that TZ q (u k — u) — ► 0. 



□ 



Remark 3. Convergence with respect to lZ q implies convergence with respect 
to the norm, which is an easy consequence of the inequality 



(En<) v ^(En 



ieN 



1/s 



(4) 



for c = (ci)igN € K N and < s < t < 00. The inequality Q easily follows for 
< \c\ s < 00 from the inequality 



E 



N 

IcL 



^E 

ieN 
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In particular, this shows that 

n q (u) > W^J^K^^W > W nlin (j2\(^^)\ 2 ) q/2 > WnunC^WuW" (5) 

for every u 6 U. Therefore, Lemma [5] implies JTOJ Lemma 4.3], where the 
authors show convergence of the sequence (ufc)fceN with respect to the norm. 

Another immediate consequence of ([5]) is the weak coercivity of the func- 
tional lZ q . ■ 

Lemma 4. Let (wfe)fcgN C dom(F) and (vk)keK C V. Assume that the sequence 
(vk)keN is bounded in V and that there exist a > and M > such that 
1~av k ( u k) < M f or a M k G N. TTien i/iere exisf u G dom(_F) and a subsequence 
(ukj)jeN such that u\ tj u and F(uk-) — F(u). 

Proof. The coercivity of lZ q and the estimate T~P'v h (uk) > aTZ q (uk) imply that 
the sequence (uk)k&t is bounded in U. Similarly, since (vk)k£N is bounded, also 
the sequence (-F(ttfe)) fc N is bounded in V. Therefore there exist a subsequence 
(itfc. and a 6 [/, i/ e 7, such that (itfc-)jeN weakly converges to u and 
(F(ufe j ))^. gN weakly converges to y. Since F is weakly sequentially closed, it 
follows that u £ dom(F) and F(u) = y. □ 

The ideas of the following proofs are based on [HI Section 3]. Still, we 
provide short proofs, since our assumptions are slightly different from [19] . where 
weak continuity of the operator F is assumed. 

Proposition 5 (Existence). For every v 8 £ V the functional T p ' q s has a min- 
imizer in U . 

Proof. Let (uk)ken satisfy 

hm ( Ufe ) = inf{7£$(u) :u£U}. 

Lemma 0] shows that there exists a subsequence (uk)jen weakly converging to 
some u £ U such that F(itjb) — F(u). Therefore the weak sequential lower 
semi- continuity of "T 9 e implies that u is a minimizer of T^s- O 

Proposition 6 (Stability). Let (vk)keN converge to v 5 £ V and let 

u k £ argmin{T Q p ;J fc (M) : u £ U} . 

Then there exists a subsequence (u^ )jeN and a minimizer u s a of such 
that lZ q {u 5 a — Uki) — > 0. If the minimizer u s a is unique, then {uk)ken converges 
to u 8 a with respect to TZ q . 

Proof. From LemmaH]we obtain the existence of a subsequence (u^ . ) jew weakly 
converging to some u £ dom(F) such that Ffakj) — 1 F{u). Since Vk — > v 5 , it 
follows that 7™, (it) < lim inf j 7™ . {u k] ) . 

On the other hand, if u £ dom^F), then 

T™(u) = (&) > limiirfT™ (u fc ) . 
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Thus u = u s n is a minimizcr of T p,9 i . 

Now note that also T p '^ s (ukj) — * T p '^ s (u). Since both and TZ q are 
weakly sequentially lower semi-continuous, this implies that TZ q {ukj) — * !Z q (u). 
Using Lemma [21 we therefore obtain the convergence of the sequence (uk)j<=n 
with respect to lZ q . 

In case the minimizer u a is unique, the convergence of the original sequence 
(uk)ken to u s a follows from a subsequence argument. □ 

Proposition 7 (Convergence). Assume that the operator equation F(u) = v 
attains a solution in dom(7?. 9 ) and that a: K>o — > R>o satisfies 

S p 

a (5) — + and — > as <5 — * . 

a(S) 

Let (5fe — *■ and let Vk EV satisfy \\vk — v\\ < 5k- Moreover, let ak = a(Sk) and 

u k G a,rgmm{TP' q Vk (u) : u £ If} . 

Then there exist an TZ q -minimizing solution of F(u) — v and a sub- 
sequence (itfc.)jeN with H q {u^ — u^) — > 0. If the TZq-minimizing solution is 
unique, then (ufe)feeN converges to with respect to lZ q . 

Proof. Let u £ dom(7?.g) be any solution of F{u) = v. The definition of Uk 
implies that 

\\F{u k ) - v k \\ p + a k n q {u k ) < \\F(u) - v k \\ p + a k K q (u) < S p k + a k K q {u) . 
In particular ||F(ufc) — Wfc|| ™> and 

§p 

lim sup 7£ g (it fc) < 7Z q (u) + limsup — = TZ q (u) . (6) 

k k a k 

This shows that there exists M > such that T™ Vh (u k ) < M for all k e N. 
Thus Lemma H] yields a subsequence (ufc<)jeN weakly converging to some ut G 
dom(F) such that F(u fcj ) F(«t). Since ||F(« fc ,) - v|| < \\F(u kj ) - v k] \\ + 
\\vk - — v\\ — > 0, it follows that F(iJ) = w. 

The weak sequential lower semi-continuity of TZ q implies that 7Z q (w) < 
liminfj IZ^Ukj). Since |(HJ) holds for every u € dom(7?. g ) satisfying F(u) =v, it 
follows that xv is an ^-minimizing solution of F(u) = v and that TZ q (uk ) — > 
7?. g (itT). Lemma[2]now shows that (ukj)jeN converges to u> with respect to TZ q . 

Again, the convergence of the original sequence (uk)k£ti to follows from 
a subsequence argument, if the TZ q -minimizing solution is unique. □ 

In the following we write a ~ 8 s for a: (0, oo) — > (0, oo) and s > 0, if there 
exist constants C > c > and So > 0, such that c<5 s < a(S) < CS S for every 
< S < So. 

For the next result on convergence rates recall the definition of the exponent 
r in Assumption [TJ 

Proposition 8 (Convergence Rates). Let Assumption^ hold. Assume that 
v s G V satisfies \\v s — v\\ < S and u s a £ argmin{T p,9 a (u) : it G U} . For a and S 
sufficiently small we obtain the following estimates: 
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If p — 1 and a[3 2 < 1, then 

MA , , ,. 

api 1 — ct/> 



Z/p > 1, t/ien 

II 5 _ tlir <• ff' + afea +(<*&)**/!>« 

||F(u* ) - ^|| p < P*<5 P + P*a/3 2 6 + {al3 2 ) p ' 

Here, p* is the conjugate of p defined by l/p* + 1/p = 1. 
In particular, if a ~ <$ p_1 , £/ten — = 0(5 1//r ). 



Proof. Since it* minimizes , the inequality 

||F(i4) - t/f + aU q (u s a ) < ||F(«t) - </f + c^(u+) 
holds. Assumption [T] and the fact that F(it^) = i? therefore imply that 

S p > \\F(u s a ) - v 5 ^ + a(n q (u s a ) - Tl q {v))) 

> \\F(u s a ) - v s \\ p +a/h \\u s a - uty -aft - F( U t)|| 

> ||F(4) - u 5 f +oft |K - U f ir - "/3 2 ||F(u*) - v s \\ - a/3 2 5 . 

This shows the assertion in the case p — 1 . 

If p > 1, we apply Young's inequality a6 < a p /p + bP* /p„ with a = ||F(u* ) — 
v 5 \\ and b — a(3 2 . Then again the assertion follows. □ 

Remark 9. Proposition[S]shows that sparsity regularization is an exact method 
for p = 1, that is, it yields exact solutions iv for noise free data and a < l/pVa 



4 Relations to Source Conditions 

We now investigate Assumption [T] more closely and show that it is indeed a 
generalization of commonly imposed source conditions involving the Bregman 
distance defined by the functional lZ q (see e.g. 0E5]). The basis of these results 
is the following lemma, which relates the Bregman distance to the squared norm 
on U in case q > 1. This result is a consequence of a special case of [3J Lemma 
2.7] (see also Corollary 3.7]). 

From now on we assume that (^i)ieN is an orthonormal basis. 

Lemma 10. Let 1 < q < 2. There exists a constant c q > only depending on 
q such that 

„ ll„~ „,I|2 

T>b{u, u) := lZ q (u) — lZ q (u) — (dlZ q (u), u — u) > 



3l«min + 2TZ q (u) + TZ q (u) 



for all u, u € dom(7?.g) for which dlZ q (u) ^ 7 which is equivalent to the as- 
sumption that J2i£fq w f I (4>ii u ) I 2 *- 9 " 1 -* < oo. 



7 



Proof. There exists d q > such that 

d q \a - b\ 2 < (\a\ 2 -i + \a- b\ 2 ~«) [|6|« - |o|« - q\a\^ sgn(a) (b - a)] (7) 

for alia, 6 €R O §5, Eq. 1]. 
Let a^iie dora(1Zq). Then 

&R q (u) = ^qwi | u) sgn((<^, u)) 0j 

provided that dlZ q (u) ^ 0. Applying (|7|), we see that 

1Z q (u) — 1Z q {u) — (dlZ q , u — u) 

= m [|<& , u) | « - | «) | « - g I u) | sgn«^ , u) ) < & , u - u) 



> 



«Jt|(0t,{t-'U)| 5 



(8) 



^ w min ~ — \ I 2 

" ma X {|(^, U )| 2 -9 + |(^- U )| 2 -9 :i£N}^*' M " ;l 

^ d q W m j n \~^| /,/, ~ _ \ I 2 

" max{2|(^, U )| 2 -9 + |(^,«)| 2 -9:*eN}^ IWl ' W U>1 

W mm \~^\/rh ~ — \ I 2 

" 3 + max{2|(^, M )|« + |<0 l ^)|9 :ie N}^ IW ' U M;l 

3Wmin + 2^ 9 (u) + lZ q (u) ^ 

Here, the third and second to last estimates follow from the inequalities (a + 
bf-t < a 2 -i + b 2 -i and a q - 2 < 1 + a q for a, 6 > 0. Thus the assertion follows 
by setting c q := d q w^ in . □ 

Proposition 11. Let F be a bounded linear operator on U, 1 < q < 2, and 

an lZ q -minimizing solution of F(u) = v. Then Assumption^ with r = 2 is 
equivalent to the source condition 

dK q {v)) G range(F*) . (9) 

In particular, if a ~ <5 P_1 , t/ien — u^|| = 0(-\/8). 

Proof. First assume that ([5]) holds. The condition dlZq^) G range(F*) implies 
the existence of a constant (7 > such that 

|(^(u t ),u-u t )| < C||F(u-u + )|| (10) 

for all m G [/. Together with Lemma fTUl this yields the inequality 

R '<»> - R «<« , » s 3 m ,„,„ + 2 R l;»t )+R , M »» - «'ii 2 + < 8 *.(« , >. - - »*> 

a 3^ + 2 R I( a t) + R , W ''-" , '' 2 - 6 '' F "- , )'l' 
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Thus, Assumption [T] is satisfied if we choose r — 2, p = 1Z q (u') + w m i n , (3\ = 
c g /(4w min + 31l q (u^)), and f3 2 = C. 

In order to show the converse implication, let Assumption [T] be satisfied for 
r = 2, that is, there exist j3i, fa > such that 

fa\\u - v)f < U q {u) - 1Z q {v!) + fa\\F(u - v})\\ 

in a neighborhood of . Both sides of this inequality are convex functions in 
the variable u that agree for u = . This implies that the subgradient at v) of 
the left hand side, which equals zero, is contained in the subgradient at of 
the right hand side. In other words, 

e dK q {v)) + faF*d(\\F{u - «t)||) . 

Consequently the source condition (J9j> holds. □ 

The following result states that the condition proposed in [19 for obtaining 
convergence rates in the non-linear, non-smooth case also follows from Assump- 
tion [T] with exponent r = 2. 

Proposition 12. Let 1 < q < 2 and v) an lZ q -minimizing solution of F(u) — 
v. Assume that there exist < 71 < 1, 72 > 0, and p > lZ q {v)) such that 

(dn q (^),u^ -u) <-fiV B (u,u^+ l2 \\F{u)-F{^)\\ (11) 

for all u £ dom(_F) with lZ q (u) < p. Then Assumption]^ holds with r — 2. In 
particular, if a ~ 8 p ~ l , then \\u s a — u^\\ — 0(VS). 

Proof. Using (fTTj) and Lemma ITTJ1 we obtain that 

7l (K q (u) - K q (u^)) > -(1 - lx ){dfc q (v)), u~u^)~ 72 \\F{u) - F{v))\\ 

> p\\u- ut|| a - 72 \\F{u) - F(«t)|| - (1 - 7l )(TZ q (u) - 7^(«t)) , 

where $ := (1 — 71) c q / (3w m [ n + 2TZ q (u^) + p). Thus Assumption [1] follows with 
fa = £/(l + 2 7l ) and fa = 72 /(l + 2 7 i). □ 

5 Convergence Rates for Sparse Solutions 

We have seen above that appropriate source conditions imply convergence rates 
of type \^S. These rates in fact can be improved considerably, if the TZ q - 
minimizing solution is sparse with respect to (0i)i S N in the sense that the 
set 

J := {i£N: (u\&) ^0} 

is finite. 

Assumption 13. Assume that the following hold: 

1. The operator equation F{u) — v has an TZ q -minimizing solution u' that is 
sparse with respect to (</>i)i S N. 
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2. The operator F is Gateaux differentiable at u>, and for every finite set 
J C N the restriction of its derivative F'(w) to span{</>j : j G J} is 
infective. 

3. There exist 71, 72 > 0, a > 0, and p > TZ q (u') such that 

TZ q (u) - n q (v?) > 71 \\F(u) - F{v)) ~ F'(« + )(u - u + ) || - 7a||.F(u) - F(ut)|| 

(12) 

for all u G dom(F) satisfying lZ q (u) < p and \\F{u) — F{u^)\\ < a. 
We first derive a convergence rates result of order 5 x / q for q > 1. 

Theorem 14 (q > 1). Let 1 < q < 2 and assume that Assumvtion\13\ holds. 
Then for a parameter choice strategy a ~ <5 P_1 we obtain the convergence rate 

\\u S a - «t|| =0(^/9). 



Proof. We verify Assurnption[T]with r = q and appropriate constants /3i, /?2 > 0. 
Then the assertion follows from Proposition [51 

Let therefore u E U satisfy TZ q (u) < p and Hi^it) — F(tt^)|| < <r. 

Define J := {i G N : {u\4>i) 7^ 0} an d W : = spanj^ : j G J}. Since u 1 * is 
sparse, the set J is finite. Therefore, the restriction of F 1 (w) to W is injective, 
which implies the existence of a constant C > such that 

CH-F'Cut) to|| > ||to|| for all w G W . 

Now denote by ttw, t^w '■ U —> U the projections 

ir w u := y^^(pj,u) <pj , tt^u := ^(<^-,tt) ^ . 

Note that by assumption (<f)j,u>) = for every j g" J, which implies that 
= ttwu^ and tt^u^ = 0. By means of the inequality 

(a + b) q < 2 q - 1 (a q + b q ) < 2(a q + b q ) for every a, b > 

it therefore follows that 

\\u-u^\\ q < 2||7T V i/(u-M t )|| 9 + 2||7r^w|| 9 

< 2C q \\F'(u^)(Tr w (u-u 1 <))\\ q + 2\\irJ v u\\ q (13) 

< 4C q \\F'(u^)(u - u^)\\ q + 2(1 + 2C q \\F'{u^)\\ q ) \\-K^u\\ q . 



We now derive an estimate for 1 1 7r^> it 1 1 9 . Using ((3|) we see that 

i^J i£J i£J 

Since q > 1, the inequality 

«</),:, u)|« - m, u^)\ q - q\(&, u t)|9-i Bgn(K^,ut)|)(^,« - «t) >0 
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holds for all i€N. Consequently, 

5>iK&>«>l a 

= £ Wi [| «)|« - | t*t)|9 - g| U t) Sgn (| , „t)|) „ _ „t) 

< *i [l <&> u)\ q - | w f ) I 9 - g| ut) |«" 1 agn(| «t)|) u - u t) 
= 7£ g (u) - ^ 9 (u f ) - (dTlgiu 1 ), u - u f ) . 

(15) 

From (|12p we obtain by considering u = u T + tu, dividing by t, and passing 
to the limit t — > that 

(dTZ q (u^), u) > - 72 ||i ;i '(u t )u|| for all u € 17 . (16) 

Together with (fT2"|) this implies the inequality 

7e g (w) - n q (u i ) - (dn g (u i ),u- u f ) 

< - 7J,(ut) + 72 ||fV)(" - u t)|| 

< K q (u) - K q {u^)+ l2 \\F{u) - F{v))\\ 

+ ~t2\\F{u)-F{v))-F l {v))[u-v))\\ 

< (1 + l2hi){K q {u) - K q {v))) + 72 (1 + 72hi)\\F(v) - F(«t)|| . 

Combination of estimates ([Tl)) - (fT7|) yields 

w min \\Tr^u\\ 9 < (1 + 72 /7i)(^ 9 N - ^(u*)) + 72(1 + 72 / 7l )||F( U ) - F(v?)\\ . 

(18) 

It remains to find an estimate for \\F'(u^)(u — u')\\ q . Since by assumption 
n q (vt), K q (u) < p, and \\F(u) - F(vt)\\ < a, it follows from (TTJ) that 

\\F'(u^(u-u^\\ q 

< 2i- 1 \\F(u) - F(v?) - F'(u f )(u - u^f + 2' ? - 1 || j P(m) - F(«t)||« 

< —{K q (u) - K q (u^+ l2 \\F(u) - F(u^\\j + 2i- 1 \\F(u) - F(u^)\\ q 



7? 



(ll q {u)-1l q {u!) 



+ ( 2 9-v,-i + » £±W ^ ) || F(U) - F(u t)|| . 

(19) 

Combining the inequalities (1 13[) . (fT5| , and (|19p. we obtain the assertion. □ 



The argumentation in the proof of Theorem [TJ] cannot be applied directly 
to the case q = 1. The main difficulty is that here the estimate (fT6"|) does not 
follow from (fT2|) . since the subgradient of 72 1 is not single valued. Therefore it 
is necessary to postulate the existence of a subgradient element £ G <972i(itt) for 
which such an inequality holds. 
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Theorem 15 (q = 1). Let 5 = 1 and assume that Assumvtion 1 13\ holds. In 

addition we assume the existence of £ E d1Zi(v/) and 73 > such that 

fti(u) - fti(«t) > -7&(e,u- ut) - 72 ||F( U ) - F(«t)|| (20) 

/or all u 6 dom(F) loit/j TZ\(u) < p and \\F{u) — F(u^)\\ < a. 
Then it follows for a parameter choice strategy a ~ 8 p ~ l that 

\\u 5 a -J\\ = 0{8). 



Proof. We show that Assumption [T] holds with r = 1 . Then the result follows 
from Proposition [S] 

Define J := {i e N : l(&>£)l — ^min} and W :— span{ </>.,■ : j 6 J}. Since 
£ € £7, it follows that J is a finite set. Therefore there exists C > such that 
C\\F'(vt)w\\ > \\w\\ for all we W. 

By assumption we have that (<pi,w) — for every i $ J. Proceeding as in 
the proof of Theorem [HI we obtain that 

||u-«t|| < C\\F'(u^(u-u^\\ + (l + C||fV)||)||4ru|| • 

Denote now m := max{|(^>i, £)| '■ * ^ which is well-defined, as [((pi, £)) ieN S 
Z 2 and therefore converges to zero. Using the inequalities < m < w m in and 
< the assumption £ € 97?.i(mT), and (f20|) . we can therefore estimate 

ii^«h = (x;k^«)i 2 ) 1/2 <ek^">i 

< y2(m - m)\{(f>i,u)\ 

• ^(Wi\{<t»i,u)\- 

' ^2( w i\(<f>h u )\ - W i |(^,U t )| - (<k,£)(0i,U-ut)) 

i6N 



< 

Here, the third to last line follows from the definition of the subgradient and 
the fact that (fa, vf) = for i £ J. 

For ||.F / (u')('U — we obtain from (fl2"|) the estimate 

\\F'(u^)(u-u^)\\ < \\F{u) -F(vt) - F'^Xu- w^j + ||F(u) - 

< 7 r 1 (^i(u)-W 1 (ut)) + (l + 72 / 7l )||F(u)-F(u t )|| . 

Again, the assertion follows by collecting the above inequalities. □ 

Remark 16. Note that in fact for the convergence rates to hold the injectivity 
of F'(u') is only required on the subspace W defined in the proofs of Theo- 
rems Q3] and [151 ■ 





m 


1 




^min 


m 


1 






m 


1 






m 


1 




^min 


rn 
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Remark 17. Consider now the special case, where F: U — ► V is linear and 
bounded. Then ([12"]) with 1 < g < 2 is equivalent to the source condition 

d1Z q (u^) n range(F*) ^ . 

Indeed, in this case the operator F equals its differential and therefore (fT2"|) 
reads as 

1l q (u)-n q (v!) > - l2 \\F{u-u^\\ , (21) 
which is equivalent to the existence of some u> E dH. q (u*) satisfying 

(w, U — U') > — 72||i r ('U — u ')\\ . 

This last inequality is in turn equivalent to the condition dlZ q (u')r\Ta,nge(F*) ^ 
0, which shows the assertion. 

In the case q = 1 the inequality (f2T)|) with 73 = 1/2 follows from (piT]) , since 

ftiM-ftiM + !<£,«-«*) > ^(^(n)-^^)) > -^||F(u-«t)|| . 

As a consequence, the convergence rate Oid 1 ^) follows from the range condi- 
tion dH q (v)) n range(F*) 7^ and the finite basis injectivity property, which 
postulates the injectivity of the restriction of F to every subspace of U spanned 
by a finite number of basis elements (f>i . ■ 

6 Conclusion 

We have studied the application of Tikhonov regularization with l q type penalty 
term for 1 < q < 2 to sparse regularization. In general, quadratic and l q 
regularization enjoy the same basic properties concerning existence, stability, 
and convergence of the corresponding approximate solutions. If additionally q 
is strictly greater than one, then also the same convergence rates can be obtained 
provided a source condition holds. 

For linear operators F this condition requires the subgradient of the penalty 
term to be contained in the range of the adjoint of F . This assumption implies 
convergence rates with respect to the Bregman distance, which for non-quadratic 
functionals in general cannot be compared with the norm on the Hilbert space. 
In the l q case, however, such a comparison is possible and leads to convergence 
rates of order y/S in the norm. 

Even better results hold if the true solution u' of the considered problem 
is known to have a sparse representation in the chosen basis. Then the l q reg- 
ularization method yields rates of order S 1 ^ , as long as the derivative of the 
operator F at is injective on the subspace spanned by the non-zero compo- 
nents of . For q = 1 and an additional assumption concerning the subgradient 
of the penalty term, this implies linear convergence of the regularized solutions 
to w . 
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